Field of the Invention
The present invention relates in general to the field of analyzing data, and more particularly to a universal actor correlator to precorrelate data for analysis and visualization.
Description of the Related Art
Enterprises collect and use data for a wide variety of purposes. Large stores of data do little good, however, unless the data is analyzed to find relevant relationships. Even if relevant relationships are found, presentation of the relationships and their impact tend to cause confusion unless some explanation of the relevance is provided. Examples of large data stores include weather sensor data, oil field surveys, and network security monitoring. One option for analysis of large data stores is perform a massive data crunch with supercomputers, however, this is expensive and often produces stale results by the time the numbers are crunched.
In the field of network security in particular, timely data analysis presents a challenge since delayed analysis tends to fail to identify threats rapidly enough to react before damage is done. Network security attackers are often organized into factions that have common political or financial goals. Attack tools are often well-developed with significant man-hours in their creation, testing and refinement. Attacks against high-value vertical targets, such as financials, power grids, military defense, etc. . . . , are generally thoroughly planned and tested. Failure to rapidly detect and respond to an attack can result in costly losses.
Generally, network security attacks are monitored and detected with specific use cases applied to individual data sources. This generally requires an intimate relationship between each specific use case and the individual data sources. For instance, each use case typically includes a specialized correlation to be written that accounts for quirks in the data. A “store now, analyze later” approach adds processing burdens for time correlations when searching for relationships, resulting in a slow and laborious analysis in which only a handful of correlations can run simultaneously, the analysis does not scale and visualizations of relationships are difficult. Applying use cases to individual data sources tends to provide a narrow perspective that is not well suited for discovery of leads and relate activity with visualizations that have a flat view showing a cascade of events. Analysis of use case output tends to be a manual, iterative and time-consuming process that depends upon individual analyst experience to flag network security attacks.